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Abstract. We provide an overview of theories of continuous time com- 
putation. These theories allow us to understand both the hardness of 
questions related to continuous time dynamical systems and the compu- 
tational power of continuous time analog models. We survey the existing 
models, summarizing results, and point to relevant references in the lit- 
erature. 

1 Introduction 

Continuous time systems arise as soon as one attempts to model systems that 
evolve over a continuous space with a continuous time. They can even emerge as 
natural descriptions of discrete time or space systems. Utilizing continuous time 
systems is a common approach in fields such as biology, physics or chemistry, 
when a huge population of agents (molecules, individuals, . . . ) is abstracted into 
real quantities such as proportions or thermodynamic data [Hirsch et al., 2003], 
[Murray, 2002]. 

There are several approaches that have led to theories on continuous time 
computations. We will explore in greater depth two primary approaches. One, 
which we call inspired by continuous time analog machines, has its roots in 
models of natural or artificial analog machinery. The other, we refer to as inspired 
by continuous time system theories, is broader in scope. It comes from research 
on continuous time systems theory from a computational perspective. Hybrid 
systems and automata theory, for example, are two sources. 

A wide range of problems related to theories of continuous time compu- 
tations are encompassed by these two approaches. They originate in fields as 
diverse as verification (see e.g. [Asarin et al., 1995]), control theory (see e.g. 
[Branicky, 1995b]), VLSI design (see e.g. [Mills, 1995], [Mills et al., 2005]), neu- 
ral networks (see for example [Orponen, 1997]) and recursion theory on the reals 
(see e.g. [Moore, 1996]). 

At its beginning, continuous time computation theory was mainly concerned 
with analog machines. Determining which systems can actually be considered 



as computational models is a very intriguing question. This relates to the philo- 
sophical discussion about what is a programmable machine, which is beyond 
the scope of this chapter. Nonetheless, there are some early examples of built 
analog devices that are generally accepted as programmable machines. They in- 
clude Bush's landmark 1931 Differential Analyzer [Bush, 1931], as well as Bill 
Phillips' Finance Phalograph, Hermann's 1814 Planimeter, Pascal's 1642 Pasca- 
line, or even the 87 b.c. Antikythcra mechanism: see [Coward, 2006]. Continuous 
time computational models also include neural networks and systems that can 
be built using electronic analog devices. Since continuous time systems are con- 
ducive to modeling huge populations, one might speculate that they will have 
a prominent role in analyzing massively parallel systems such as the Internet 
[Papadimitriou, 2001]. 

The first true model of a universal continuous time machine was proposed 
by Shannon [Shannon, 1941], who introduced it as a model of the Differential 
Analyzer. During the 1950s and 60s an extensive body of literature was pub- 
lished about the programming of such machines . There were also a number of 
significant publications on how to use analog devices to solve discrete or contin- 
uous problems: see e.g. [Vergis et al., 1986] and the references therein. However, 
most of this early literature is now only marginally relevant given the ways in 
which our current understanding of computability and complexity theory have 
developed. 

The research on artificial neural networks, despite the fact that it mainly 
focused on discrete time analog models, has motivated a change of perspective 
due to its many shared concepts and goals with today's standard computability 
and complexity theory [Orponen, 1997], [Orponen, 1994]. Another line of devel- 
opment of continuous time computation theory has been motivated by hybrid 
systems, particularly by questions related to the hardness of their verification 
and control: see for example [Branicky, 1995b] and [Asarin et al., 1995]. 

In recent years there has also been a surge of interest in alternatives to 
classical digital models other than continuous time systems. Those alterna- 
tives include discrete-time analog-space models like artificial neural networks 
[Orponen, 1997], optical models [Woods and Naughton, 2005], signal machines 
[Durand-Lose, 2005] and the Blum Shub and Smalc model [Blum et al., 1998]. 
More generally there have also been many recent developments in non-classical 
and more-or-less realistic or futuristic models such as exotic cellular automata 
models [Grigoricff and Margenstern, 2004], molecular or natural computations 
[Head, 1987], [Adleman, 1994], [Lipton, 1995], [Paun, 2002] , black hole computa- 
tions [Hogarth, 1992], or quantum computations [Deutsch, 1985], [Gruska, 1997], 
[Shor, 1994], [Kieu, 2004]. Some of these contributions are detailed in this vol- 
ume. 



5 See for example the very instructive Doug Coward 's web Analog Computer Museum 
[Coward, 2006] and its bibliography. This literature reveals a quite forgotten art of 
programming continuous time and hybrid (digital-analog) machines, which level of 
sophistication is close to today's engineering programming. 
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The computational power of discrete time models are fairly well known 
and understood thanks in large part to the Church- Turing thesis. The Church- 
Turing thesis states that all reasonable and sufficiently powerful models are 
equivalent. For continuous time computation, the situation is far from being 
so clear, and there has not been a significant effort toward unifying concepts. 
Nonetheless, some recent results establish the equivalence between apparently 
distinct models [Graga and Costa, 2003], [Graga, 2004], [Graga et al., 2005], and 
[Bournez et al., 2007], which give us hope that a unified theory of continuous 
time computation may not be too far in the future. 

This text can be considered an up-to-date version of Orponen's 1997 survey 
[Orponcn, 1997]. Orponen states at the end of his introduction of that the effects 
of imprecision and noise in analog computations are still far from being under- 
stood and that a robust complexity theory of continuous time models has yet 
to be developed. Although this evaluation remains largely accurate with regard 
to imprecision and noise, we will see in the present survey that in the inter- 
vening decade much progress in understanding the computability and even the 
complexity of continuous time computations has been made. 

This chapter is organized as follows. In Section 2, we review the most relevant 
continuous time models. In sections 3 and 4 we discuss, respectively, computabil- 
ity and complexity issues in continuous time computations. In these sections we 
focus mainly on continuous time dynamical systems. In Section 5 we address 
the effect of imprecision and noise in analog computations. Finally, in Section 6, 
we conclude with some general insights and directions of further research in the 
field of continuous time computation. 

2 Continuous Time Models 

With a historical perspective in mind, we outline in this section several of the ma- 
jor classes of continuous time models that motivated interest in this field These 
models also illustrate concepts like continuous dynamics and input /output. 

2.1 Models inspired by analog machines 

GPAC and other circuit models Probably, the best known universal continu- 
ous time machine is the Differential Analyzer, built at MIT under the supervision 
of Vannevar Bush [Bush, 1931] for the first time in 1931. The idea of assembling 
integrator devices to solve differential equations dates back to Lord Kelvin in 
1876 [Thomson, 1876]. Mechanical 6 , and later on electronic, differential analyz- 
ers were used to solve various kinds of differential equations primarily related to 
problems in the field of engineering: sec for e.g. [Bowles, 1996], or more generally 
[Williams, 1996] for historical accounts. By the 1960s differential analysers were 
progressively discarded in favor of digital technology 

6 And even MECANO machines: see [Bowles, 1996]. 
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The first theoretical study of the computational capabilities of continuous 
time universal machines was published by Shannon. In [Shannon, 1941], he pro- 
posed what is now referred to as the General Purpose Analog Computer (GPAC) 
as a theoretical model of Vannevar Bush's differential analyzer. The model, 
later refined in the series of papers [Pour-El, 1974], [Lipshitz and Rubel, 1987], 
[Graca and Costa, 2003] , [Graca, 2004] , consists of families of circuits built with 
the basic units presented in Figure 1. There are some restrictions to the kinds of 
intcrconnectivity that are allowed to avoid undesirable behavior: e.g. non-unique 
outputs. For further details and discussions, refer to [Graga and Costa, 2003] and 
[Graca, 2002]. 

Shannon, in his original paper, already mentions that the GPAC generates 
polynomials, the exponential function, the usual trigonometric functions, and 
their inverses (see Figure 2). More generally, he claimed in [Shannon, 1941] that 
a function can be generated by a GPAC iff it is differentially algebraic, i.e. it 
satisfies some algebraic differential equation of the form 



where p is a non-zero polynomial in all its variables. As a corollary, and noting 
that the Gamma function r(x) — J °° t x ~ 1 e~ t dt or the Riemann's Zeta function 
C( x ) = Sfclo W are n °t d- a - [Rubel, 1989], it follows that the Gamma and the 
Zeta functions are examples of functions that can not be generated by a GPAC. 

However, Shannon's proof relating functions generated by GPACs with dif- 
ferentially algebraic functions was incomplete (as pointed out and partially cor- 
rected by [Pour-El, 1974], [Lipshitz and Rubel, 1987]). However, for the more 
robust class of GPACs defined in [Graga and Costa, 2003], the following stronger 
property holds: a scalar function / : R — > R is generated by a GPAC iff it is a 
component of the solution of a system y' = p(t, y), where p is a vector of polyno- 
mials. A function / : R — > M fe is generated by a GPAC iff all of its components 
are. 

The r function is indeed GPAC computable, if a notion of computation 
inspired from recursive analysis is considered [Graga, 2004]. GPAC computable 
functions in this sense correspond precisely to computable functions over the 
reals [Bournez et al., 2007]. 

Rubel proposed [Rubel, 1993] an extension of Shannon's original GPAC. 
In Rubel's model, the Extended Analog Computer (EAC), operations to solve 
boundary value problems or to take certain infinite limits were added. We refer 
to [Mills, 1995] and [Mills ct al., 2005] for descriptions of actual working imple- 
mentations of Rubel's EAC. 

More broadly, a discussion of circuits made of general basic units has been 
presented recently in [Tucker and Zucker, 2007] . Equational specifications of such 
circuits, as well as their semantics, are given by fixed points of operators over 
the space of continuous streams. Under suitable hypotheses, this operator is 
contracting and an extension of Banach fixed point theorem for metric spaces 
guarantees existence and unicity of the fixed point. Moreover, that fixed point 
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can also be proved to be continuous and concretely computable whenever the 
basic modules also have those properties. 



Hopfleld network models Another well known continuous time model is the 
"neural network" model proposed by John Hopfleld in 1984 in [Hopfleld, 1984]. 
These networks can be implemented in electrical [Hopfleld, 1984] or optical hard- 
ware [Stoll and Lee, 1988]. 

A symmetric Hopfleld network is made of a finite number, say n, of simple 
computational units, or neurons. The architecture of the network is given by 
some (non oriented) graph whose nodes are the neurons, and whose edges are 
labeled by some weights, the synaptic weights. The graph can be assumed to be 
complete by replacing the absence of a connection between two nodes by an edge 
whose weight is null. 

The state of each neuron i at time t is given by some real value Ui(t). Starting 
from some given initial state «o € M. n , the global dynamic of the network is 
defined by a system of differential equations 

C i u' i (t) = Y,W i , j V j -u i /R i + I i , 

3 

where Vi = o~{ui), a is some saturating function such as a(u) = atanu + (3, 
Wij = Wj_i is the weight of the edge between i and j, Ci,I i} Ri are some con- 
stants [Hopfleld, 1984]. 

Hopficld proved in [Hopficld, 1984], by a Lyapunov-function argument, that 
such systems are globally asymptotically stable, i.e. from any initial state, the sys- 
tem relaxes toward some stable equilibrium state. Indeed, consider for example 
the energy function [Hopficld, 1984] 

^ = - 2 E E w«W + E i P °-Hv)dv + £ im. 

i j i 1 " i 

The function E is bounded and its derivative is negative. Hence the time evo- 
lution of the whole system is a motion in a state space that seeks out (possibly 
local) minima of E. 

This convergence behavior has been used by Hopfleld to explore various ap- 
plications such as associative memory, or to solve combinatorial optimization 
problems [Hopfield, 1984], [Hopfleld and Tank, 1985]. 

An exponential lower bound on the convergence time of continuous time Hop- 
ficld networks has been related to their dimension in [Sfma and Orponen, 2003a]. 
Such continuous time symmetric networks can be proved to simulate any finite 
binary-state discrete-time recurrent neural network [Orponen and Sfma, 2000], 
[Sfma and Orponen, 2003b] . 

Networks of spiking neurons If one classifies, following [Maass, 1997b], neu- 
ral network models according to their activation functions and dynamics, three 
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different generations can be distinguished. The first generation, with discontin- 
uous activation functions, includes multilayer perceptrons, Hopfield networks, 
and Boltzmann machines (see for example [Abdi, 1994] for an introduction to all 
mentioned Neural Networks models). The output of this generation of networks 
is digital. The second generation of networks use continuous activation functions 
instead of step or threshold functions to compute the output signals. These in- 
clude feedforward and recurrent sigmoidal neural network, radial basis functions 
networks, and continuous time Hopfield networks. Their input and output is ana- 
log. The third generation of networks is based on spiking neurons and encodes 
variables in time differences between pulses. This generation exhibits continuous 
time dynamics and is the most biologically realistic [Maass and Bishop, 1998]. 

There are several mathematical models of spiking neurons of which we will 
focus on one, whose computational properties have been investigated in depth. 
The Spiking Neural Network model is represented by a finite directed graph. To 
each node v (neuron) of the graph is associated a threshold function 9 V : R + — ► 
1U {oo}, and to each edge (u, v) (synapse) is associated a response-function 
e UiV : R + — > R and a weight-function w UjV . 

For a non-input neuron v, one defines its set F v of firing times recursively. 
The first element of F v is inf{t\P v (t) > 9 V (0)}, and for any s £ F v , the next 
larger element of F v is inf{£|i > s and P v (t) > 8 v (t — s)} where 



The above can be replaced by some bias function. We use it here to guar- 
antee that P v is well-defined even if F u = for all u with w u . v ^ 0. To ap- 
proximate biological realism, restrictions are placed on the allowed response- 
functions and bias-functions of these models: see [Maass, 1997b], [Maass, 1999], 
[Natschlager and Maass, 2002], or [Maass, 2002], [Maass, 2003], for discussions 
on the model. In particular, rapidly fading memory is a biological constraint 
which prevents chaotic behavior in networks with a continuous time dynamic. 
Recently, the use of feedback to overcome the limitations of such a constraint 
was analized in [Maass et al., 2007]. 

The study of the computational power of several variants of spiking neural 
networks was initiated in [Maass, 1996a]. Noisy extensions of the model have 
been considered [Maass, 1996b], [Maass, 1997a], [Maass and Natschlager, 2000]. 
A survey of complexity results can be found in [Sfma and Orponen, 2003c]. Re- 
strictions that are easier to implement in hardware versions have also been in- 
vestigated in [Maass and Ruf, 1999]. 

R-recursive functions Moore proposed a theory of recursive functions on 
the reals in [Moore, 1996], which is defined in analogy with classical recursion 
theory and corresponds to a conceptual analog computer operating in continuous 
time. As we will see, this continuous time model has in particular the capability 
of solving differential equations, similar to an idealized analog integrator of the 
GPAC. In fact, the theory of R-recursive functions can be seen as an extension of 





u seF u ,s<t 
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Shannon's theory for the GPAC. A general discussion of the motivations behind 
M-recursion theory can be found in [Mycka and Costa, 2005]. 

A function algebra [B\, B2, 0±, O2, • •■] is the smallest set containing basic 
functions {-Bi, B2, ■■■} and closed under certain operations {0\,02, ■■■}, which 
take one or more functions in the class and create new ones. Although function 
algebras have been defined in the context of recursion theory on the integers, and 
widely used to characterize computability and complexity classes [Clote, 1998], 
they are equally suitable to define classes of real valued recursive functions. 

The R-recursive functions were first defined in [Moore, 1996]. These are given 
by the function algebra M. — [0, 1, U; comp, int, minim], 7 where U is the set of 
projection functions Ui(x) = Xj, comp is composition, int is an operation that 
given / and g returns the solution of the initial value problem h(x, 0) = f(x) 
and d y h(x,y) = g(x,y,h) and minim returns the smallest zero /i y f(x,y) of a 
given /. Moore also studied the weaker algebra X = [0, 1, — 1, U ; comp, int] and 
claimed its equivalence with the class of unary functions generated by the GPAC 
[Moore, 1996]. 

Many non recursively enumerable sets are M-recursive. Since minim is the 
operation in M. which gives rise to uncomputable functions, a natural question 
is to ask if minim can be replaced by some other operation of mathematical 
analysis. This was done in [Mycka and Costa, 2004] where minim is replaced by 
the operation lim, which returns the infinite limits of the functions in the algebra. 
These authors stratify [0, 1, — 1, U; comp, int, lim] according to the allowed number 
(77) of nested limits and relate the resulting ry-hierarchy with the arithmetical 
and analytical hierarchies. In [Loff et al., 2007a] it is shown that the ^-hierarchy 
does not collapse (see also [Loff, 2007]), which implies that infinite limits and 
first order integration are not interchangeable operations [Loff et al., 2007b]. 

The algebra X only contains analytic functions and is not closed under iter- 
ation [Campagnolo et al., 2000]. However, if an arbitrarily smooth extension to 
the reals 9 of the Heaviside function is included in the set of basic functions of 
X, then X+9 contains extensions to the reals of all primitive recursive functions. 

The closure of fragments of X + 9 = [0, 1, — 1, 6, U; comp, int] under dis- 
crete operations like bounded products, bounded sums and bounded recursion, 
has been investigated in the thesis [Campagnolo, 2001] and also in the papers 
[Campagnolo et al., 2002], [Campagnolo, 2002], [Campagnolo, 2004]. 

In particular, several authors studied the function algebra C — [0, 1, — 1, tt, 9, U; 
comp, LI] where the LI is only able to solve linear differential equations (i.e., it 
restricts int to the case d y h(x,y) = g(x,y) h(x,y)). The class C contains exten- 
sions to the reals of all the elementary functions, [Campagnolo et al., 2002]. 

Instead of asking which computable functions over N have extensions to K in 
a given function algebra, Bournez and Hainry consider classes of functions over 
R computable according to recursive analysis, and characterize them precisely 
with function algebras. This was done for the elementarily computable functions 
[Bournez and Hainry, 2005], characterized as C closed under a restricted limit 

7 We consider that the operator int preserves analyticity (see 
[Campagnolo et al., 2000], [Campagnolo, 2002]). 
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schema. This was extended to yield a characterization of the whole class of com- 
putable functions over the reals [Bournez and Hainry, 2006] , adding a restricted 
minimisation schema. Those results provide syntactical characterizations of real 
computable functions in a continuous setting, which is arguably more natural 
than the higher order Turing machines of recursive analysis. 

A more general approach to the structural complexity of real recursive classes, 
developed in [Campagnolo and Ojakian, 2007], is based on the notion of approx- 
imation. This notion was used to lift complexity results from N to R, and was 
applied in particular to characterize C 

Somewhat surprisingly, the results above indicate that two distinct models 
of computation over the reals (computable analysis and real recursive functions) 
can be linked in an elegant way. 

2.2 Models inspired by continuous time system theories 

Hybrid Systems An increasing number of systems exhibit some interplay be- 
tween discrete and analog behaviors. The investigation of these systems has led 
to relevant new results about continuous time computation. 

A variety of models have been considered: see for example the conference 
series Hybrid Systems Computation and Control or [Branicky, 1995a]. However, 
hybrid systems 8 are essentially modeled either as differential equations with dis- 
continuous right hand sides, as differential equations with continuous and dis- 
crete variables, or as hybrid automata. A hybrid automaton is a finite state 
automaton extended with variables. Its associated dynamics consists of guarded 
discrete transitions between states of the automaton that can reset some vari- 
ables. Typical properties of hybrid systems that have been considered are reach- 
ability, stability and controllability. 

With respect to the differential equation modeling approach, Branicky proved 
in [Branicky, 1995b] that any hybrid system model that can implement a clock 
and also implement general continuous ordinary differential equations is able to 
simulate Turing machines. Asarin, Maler and Pnueli proved in [Asarin et al., 1995] 
that piecewise constant differential equations can simulate Turing machines in 
M 3 , while the reachability problem for these systems in dimension d < 2 is de- 
cidable [Asarin et al., 1995]. Piecewise constant differential equations, as well 
as many hybrid systems models, exhibit the so-called Zeno's phenomenon: an 
infinite number of discrete transitions may happen in a finite time. This has 
been used in [Asarin and Maler, 1998] to prove that arithmetical sets can be 
recognized in finite time by these systems. Their exact computational power 
has been characterized in terms of their dimension in [Bournez, 1999a] and 
[Bournez, 1999b]. The Jordan's theorem based argument of [Asarin et al., 1995] 
to get decidability for planar piecewise constant differential equations has been 

8 "Hybrid" refers here to the fact that the systems have intermixed discrete and con- 
tinuous evolutions. This differs from historical literature about analog computations, 
where "hybrid" often refers to machines with a mixture of analog and digital com- 
ponents. 
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generalized for planar polynomial systems [Ceraens and Viksna, 1996] and for 
planar differential inclusion systems [Asarin et al., 2001]. 

There is extensive literature on the hybrid automata modeling approach 
about determining the exact frontier between decidability and non-decidability 
for reachability properties, according to the type of allowed dynamics, guards, 
and resets. The reachability property has been proved decidable for timed au- 
tomata [Alur and Dill, 1990]. By reduction to this result, or by a finite bisimu- 
lation argument in the same spirit, this has also been generalized to multirate 
automata [Alur et al., 1995], to specific classes of updatable timed automata in 
[Bouyer et al., 2000a], [Bouyer et al., 2000b], and to initialized rectangular au- 
tomata in [Henzinger et al., 1998], [Puri and Varaiya, 1994]. There is a multi- 
tude of undecidability results, most of which rely on simulations of Minsky two- 
counter machines. For example, the reachability problem is semi-decidable but 
non-decidable for linear hybrid automata [Alur et al., 1995], [Nicollin et al., 1993] 
The same problem is known to be undecidable for rectangular automata with at 
least 5 clocks and one two-slope variable [Henzinger et al., 1998], or for timed au- 
tomata with two skewed clocks [Alur et al., 1995]. For discussion of these results, 
see also [Asarin and Schneider, 2002]. Refer to [Blondcl and Tsitsiklis, 1999], and 
[Collins and van Schuppen, 2004] or to the survey [Blondel and Tsitsiklis, 2000] 
for properties other than reachability (for example, stability and observability). 

O-minimal hybrid systems are initialized hybrid systems whose relevant sets 
and flows are definable in an o-minimal theory. These systems always admit a 
finite bisimulation [Lafferriere and Pappas, 2000]. However, their definition can 
be extended to a more general class of "non-deterministic" o-minimal systems 
[Brihaye and Michaux, 2005], for which the reachability problem is undecidable 
in the Turing model, as well as in the Blum Shub Smale model of compu- 
tation [Brihaye, 2006]. Upper bounds have been obtained on the size of the 
finite bisimulation for Pfaffian hybrid systems [Korovina and Vorobjov, 2004] 
[Korovina and Vorobjov, 2006] using the word encoding technique introduced in 
[Brihaye and Michaux, 2005]. 



Automata theory There have been several attempts to adapt classical discrete 
automata theory to continuous time; this is sometimes referred to as the general 
program of Trakhtcnbrot [Trakhtcnbrot , 1995]. 

One attempt is related to timed automata, which can be seen as languages 
recognizers [Alur and Dill, 1994]. Many specific decision problems have been con- 
sidered for timed automata: see survey [Alur and Madhusudan, 2004]. Timed 
regular languages are known to be closed under intersection, union, and re- 
naming, but not under complementation. The membership and empty language 
problems are decidable, whereas inclusion and universal language problems are 
undecidable. The closure of timed regular languages under shuffling is inves- 
tigated in [Finkel, 2006]. Several variants of Kleene's theorem are established 
[Asarin et al., 1997], [Asarin, 1998], [Asarin et al., 2002], [Bouyer and Petit, 1999], 
[Bouyer and Petit, 2002], and in [Asarin and Dima, 2002]. There have been some 
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attempts to establish pumping lemmas [Bcauquicr, 1998]. A review, with discus- 
sions and open problems related to this approach can be found in [Asarin, 2004]. 

An alternative and independent automata theory over continuous time has 
been developed in [Rabinovich and Trakhtenbrot, 1997], [Trakhtenbrot, 1999], 
[Rabinovich, 2003]. Here automata are not considered as language recognizers, 
but as computing operators on signals. A signal is a function from the non- 
negative real numbers to a finite alphabet (the set of the channel's states). Au- 
tomata theory is extended to continuous time, and it is argued that the behavior 
of finite state devices is ruled by so-called finite memory retrospective functions. 
These are proved to be speed-independant, i.e. independant under "stretchings" 
of the time axis. Closure properties of operators on signals are established, and 
the representation of finite memory retrospective functions by finite transition 
diagrams (transducers) is discussed. See also [Francisco, 2002] for a detailed pre- 
sentation of Trakhtenbrot and Rabinovich's theory, and for discussions about 
the representation of finite memory retrospective operators by circuits. 

Finally, another independent approach is considered in [Ruohonen, 2004], 
where Chomsky-like hierarchies are established for families of sets of piecewise 
continuous functions. Differential equations, associated to specific memory struc- 
tures, are used to recognize sets of functions. Ruohonen shows that the resulting 
hierarchies are not trivial and establishes closure properties and inclusions be- 
tween classes. 



2.3 Other computational models 

In addition to the two previously described approaches, there are a number 
of other computational models that have lead to interesting developments in 
continuous time computation theory. 

The question of whether Einstein's general relativity equations admit space- 
time solutions that allow an observer to view an eternity in a finite time was 
investigated and proved possible in [Hogarth, 1992]. The question of whether 
this implies that super-tasks can in principle be solved has been investigated in 
[Earman and Norton, 1993], [Hogarth, 1996], [Hogarth, 1994], [Hogarth, 2006], 
[Etesi and Nemeti, 2002], [Nemeti and Andreka, 2006], [Nemeti and David, 2006], 
and [Welch, 2006]. 

Some machine inspired models are neither clearly digital nor analog. For ex- 
ample, the power of planar mechanisms attracted great interest in England and 
France in the late 1800s, and in the 1940s in Russia. Specifically, these consisted 
of rigid bars constrained to a plane and joined at either end by rotable rivets. A 
theorem attributed 9 to Kempe [Kempe, 1876] states that they are able to com- 
pute all algebraic functions: see for e.g. [Artobolevskii, 1964] or [Svoboda, 1948]. 



9 The theorem is very often attributed to Kempe [Artobolevskii, 1964], 
[Svoboda, 1948], even if he apparently never proved exactly that. 
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3 ODEs and properties 



Most of the continuous time models described above have a continuous dynamics 
described by differential equations. In Shannon's GPAC and Hopfield networks, 
the input corresponds to the initial condition while the output is, respectively, 
the time evolution or the equilibrium state of the system. Other models are 
language recognizers. The input again corresponds to the initial condition, or 
some initial control, and the output is determined by some accepting region in 
the state space of the system. All these systems therefore fall into the framework 
of dynamical systems. 

In this section we will recall some fundamental results about dynamical sys- 
tems and differential equations and discuss how different models can be compared 
in this general framework. 

3.1 ODEs and dynamical systems 

Let's consider that we are working in R™ (in general, we could consider any 
vector space with a norm). Let us consider / : E — ► R™, where E C R" is 
open. An ODE is given by y' = f(y) and its solution is a differentiable function 
y : I C R — > E that satisfies the equation. 

For any x e E, the fundamental existence-uniqueness theorem (see e.g. 
[Hirsch et al., 2003]) for differential equations states that if / is Lipschitz on E, 
i.e. if there exists K such that - f{y2)\\ < k\\y-i - y 2 \\ for all y lt y 2 G E, 

then the solution of 

v = f(y), y(*o) - * (l) 

exists and is unique on a certain maximal interval of existence I C R. In the 
terminology of dynamical systems, y(t) is referred to as the trajectory, R™ as 
the phase space, and the function cf>(t,x), which gives the position y(t) of the 
solution at time t with initial condition x, as the flow. The graph of y in R™ is 
called the orbit. 

In particular, if / is continuously differentiable on E then the existence- 
uniqueness condition is fulfilled [Hirsch et al., 2003]. Most of the mathematical 
theory has been developed in this case, but can be extended to weaker conditions. 
In particular, if / is assumed to be only continuous, then uniqueness is lost, but 
existence is guaranteed: see for example [Coddington and Levinson, 1972]. If / 
is allowed to be discontinuous, then the definition of solution needs to be refined. 
This is explored by Filippov in [Filippov, 1988]. Some hybrid system models use 
distinct and ad hoc notions of solutions. For example, a solution of a piecewise 
constant differential equation in [Asarin et al., 1995] is a continuous function 
whose right derivative satisfies the equation. 

In general, a dynamical system can be defined as the action of a subgroup 
T of R on a space X, i.e. by a function (a flow) <f> : T x X ^ X satisfying the 
following two equations 

4>{Q,x) = x (2) 

4>(t,<t>(8,x)) = <f>(t + 8,x). (3) 
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It is well known that subgroups T of R are either dense in R or isomorphic 
to the integers. In the first case, the time is called continuous, in the latter case, 
discrete. 

Since flows obtained by initial value problems (IVP) of the form (1) satisfy 
equations (2) and (3), they correspond to specific continuous time and space dy- 
namical systems. Although not all continuous time and space dynamical systems 
can be put in a form of a differential equation, IVPs of the form (1) arc suffi- 
ciently general to cover a very wide class of such systems. In particular, if (f> is 
continuously differentiable, then y' = f(y), with f(y) — ^(j){t,y)\ t _ Q , describes 
the dynamical system. 

For discrete time systems, we can assume without loss of generality that T is 
the integers. The analog of of IVP (1) for discrete time systems is a recurrence 
equation of type 

yt+i = f{y t ), ya = x. (4) 

A dynamical system whose space is discrete and that evolves discretely is 
termed digital, otherwise it is analog. A classification of some computational 
models according to the nature of their space and time can be found in Figure 3. 



3.2 Dissipative and non-dissipative systems 

A point x* of the state space is called an equilibrium point if f(x*) = 0. If 
the system is at x* it will remain there. It is said to be stable if for every 
neighborhood U of x* there is a neighborhood W of x* in U such that every 
solution starting from a point x of W is defined and is in U for all time t > 0. 
The point is asymptotically stable if, in addition to the properties above, we have 
limy(i) = x* [Hirsch ct al., 2003]. 

Some local conditions on the differential Df(x*) of / in x* have been clearly 
established. If at an equilibrium point x* all eigenvalues of Df(x*) have nega- 
tive real parts, then x* is asymptotically stable, and furthermore nearby solu- 
tions approach x* exponentially. In that case, x* is called a sink. At a stable 
equilibrium point x* , no eigenvalue of Df{x*) can have a positive real part 
[Hirsch et al., 2003]. 

In practice, Lyapunov's stability theorem applies more broadly (i.e., even if 
x* is not a sink). It states that if there exists a continuous function V defined 
on a neighborhood of x* , differentiable (except perhaps on x*) with V(x*) = 0, 
V(x) > for x ^ x* , and dV(x)/dt < for x ^ x* then x* is stable. If, 
in addition, dV(x)/dt < for x ^ x* , then x* is asymptotically stable: see 
[Hirsch et al., 2003]. 

If the function V satisfies the previous conditions everywhere, then the system 
is globally asymptotically stable. Whatever the initial point x is, the trajectories 
will eventually converge to local minima of V. In this context, the Lyapunov func- 
tion V can be interpreted as an energy, and its minima correspond to attractors 
of the dynamical system. These arc bounded subsets of the phase space to which 
regions of initial conditions of nonzero volume converge as time increases. 
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A constant unit 



An adder unit 



w'(t) = u{t)v'(t) u - 
w(to) — a v- 



An integrator unit 



A multiplier unit 



Fig. 1. Different types of units used in a GPAC. 



2/1=2/3 &yi(0) = l 
2/2 = Vi & 2/2(0) = 
y' 3 = - yi & y 3 (0) = 



Fig. 2. Generating cos and sin via a GPAC: circuit version on the left and ODE version 
on the right. One has yi = cos, y 2 = sin, y 3 — — sin. 



Space 

Time 


Discrete 


Continuous 


Discrete 


[Turing, 1936] machines 
[Church, 1936] lambda calculus 
[Kleene, 1936] recursive functions 
[Post, 1946] systems 
Cellular automata 
Stack automata 
Finite State Automata 


Discrete time [Hopfield, 1984] neural networks 
[Siegelmann and Sontag, 1994] neural networks 
[Asarin et al., 1995] PCD systems 
[Blum et al., 1989] machines 
[Woods and Naughton, 2005] optical machines 
[Durand-Lose, 2005] signal machines 
[Moore, 1998a] Dynamical Recognizers 


Continuous 


[Dee and Ghil, 1984] BDE models 


[Shannon, 1941] GPACs 
Continuous time [Hopfield, 1984] neural networks 
[Branicky, 1995b] hybrid systems 
[Asarin et al., 1995] PCD systems 
[Alur and Dill, 1990] timed automata 
[Moore, 1996] E-recursive functions 



Fig. 3. A classification of some computational models, according to their space and 
time. 
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A dynamical system is called dissipative if the volume of a set decreases under 
the flow for some region of the phase space. Dissipative systems are characterized 
by the presence of attractors. By opposition, a dynamical system is said to 
be volume-preserving if the volume is conserved. For instance, all Hamiltonian 
systems are volume-preserving because of Liouville's theorem [Arnold, 1989]. 
Volume preserving dynamical system cannot be globally asymptotically stable 
[Arnold, 1989]. 

3.3 Computability of solutions of ODEs 

Here we review some results on the computability of solutions of IVPs in the 
framework of recursive analysis (see e.g. [Weihrauch, 2000] and the corresponding 
chapter in this volume). 

In general, given a computable function /, one can investigate if the solution 
of the IVP (1) is also computable in the sense of recursive analysis. If we require 
that the IVP has a unique solution then that solution is computable. Formally, 
if / is computable on [0, 1] x [— 1, 1] and the IVP y' = f(t,y), y(0) = has a 
unique solution on [0, b], < b < 1, then the solution y is computable on [0, b\. 

This result also holds for a general n-dimensional IVP if its solution is unique 
[Ruohoncn, 1996]. However, computability of solutions is lost as soon as unique- 
ness of solutions is relaxed, even in dimension 1. Indeed, the famous result of 
[Pour-El and Richards, 1979] shows that there exists a polynomial-time com- 
putable function / : [0,1] x [—1,1] — > K, such that the equation y' = f(t,y), 
with y(0) = 0, has non-unique solutions, but none of them is computable on any 
closed finite time interval. 

Similar phenomena hold for other natural equations: the 3-dimensional wave 
equation (which is a partial equation), with computable initial data, can have 
a unique solution which is nowhere computable 10 [Pour-El and Richards, 1981], 
[Pour-El and Zhong, 1997]. Notice that, even if / is assumed computable and 
analytic, and the solution unique, it may happen that the maximal interval (a, (3) 
of existence of the solution is non-computable [Graca et al., 2006]. This same 
question is open if / is polynomial. Those authors show, however, that if / and 
/' are continuous and computable, then the solution of y' = f(y, t), y(0) = x, for 
computable x, is also computable on its maximal interval of existence. Refer also 
to [Pour-El and Richards, 1989], [Ko, 1983] for more uncomput ability results, 
and also to [Ko, 1983], [Ko, 1991] for related complexity issues. 

3.4 Static undecidability 

As observed in [Asarin, 1995] and in [Ruohonen, 1997b], it is relatively simple 
but not very informative to get undecidability results with continuous time dy- 
namical systems, if / encodes a undecidable problem. To illustrate this, we recall 

10 However, in all these cases, the problems under study are ill-posed: either the solution 
is not unique, or it is unstable and the addition of some natural regularity conditions 
to prevent ill-posedness do yield computability [Weihrauch and Zhong, 2002] . 
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the following example in [Ruohoncn, 1997b]. Ruohonen discusses the event de- 
tection problem: given a differential equation y' = f(t, y), with initial value y(0), 
decide if a given condition gj(t,y(t),y'(t)) = 0,j = 1, • • • , k happens at some 
time t in a given interval /. Given the Turing machine A4, the sequence /o, /i, • • • 
of rationals defined by 



is not a computable sequence of rationals, but is a computable sequence of reals, 
following the nomenclature of [Pour- El and Richards, 1989]. Now, the detection 
of the event y(t) = for the ordinary differential equation y' = 0, given n, and 
the initial value y(0) = /„, is undecidable over any interval containing 0, because 
/„ = is undecidable. 

A further modification can be obtained as follows [Ruohonen, 1997b]. He 
defines the smooth function 



which is computable on [0, oo). The detection of the event yi(t) = for the ODE 



given an initial value yi(0) = 1, 2/2(0) — n, where n is a nonnegative integer is 
then undecidable on [0, 1]. 

As put forth in [Asarin, 1995] undecidability results given by recursive anal- 
ysis are somehow built similarly. 

3.5 Dynamic undecidability 

To be able to discuss in more detail computability of differential equations, we 
will focus on ODEs that encode the transitions of a Turing machine instead 
of the result of the whole computation simulation 11 . Typically, we start with 
some (simple) computable injective function which encodes any configuration 
of a Turing machine M as a point in K". Let x be the encoding of the initial 
configuration of M.. Then, we look for a function / : E C M. n+1 — > W 1 such that 
the solution of y'(t) = f(y,t), with 2/(0) = x, at time T £ N is the encoding 
of the configuration of M after T steps. We will sec, in the remainder of this 
section, that / can be restricted to have low dimension, to be smooth or even 
analytic, or to be defined on a compact domain. 

Instead of stating that the property above is a Turing machine simulation, 
we can address it as a reachability result. Given the IVP defined by / and x, and 
any region A C K™, we are interested in deciding if there is a t > such y(t) G A, 
i.e., if the flow starting in x crosses A. It is clear that if / simulates a Turing 

11 This is called dynamic undecidability in [Ruohonen, 1993]. 




2 m if M stops in m steps on input n 
if M. does not stop on input n 



g{x) = /[x+i/2j e 



y[ = .9(2/2) - 1 
2/2 = 
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machine in the previous sense, then reachability for that system is undecidable 
(just consider A as encoding the halting configurations of A4). So, reachability is 
another way to address the computability of ODEs and a negative result is often 
a byproduct of the simulation of Turing machines. Similarly, undecidability of 
event detection follows from Turing simulation results. 

Computability of reachable and invariant sets have been investigated in 
[Collins, 2005] for continuous time systems and in [Collins and Lygeros, 2005] 
for hybrid systems. 

In general, viewing Turing machines as dynamical systems provides them 
a physical interpretation which is not provided by the von Neumann picture 
[Campagnolo, 2001]. This also shows that many qualitative features of (analog or 
non-analog) dynamical systems, e.g. questions about basins of attraction, chaotic 
behavior or even periodicity, are non computable [Moore, 1990]. Conversely, this 
brings into the realm of Turing machines and computability in general questions 
traditionally related to dynamical systems. These include in particular the re- 
lations between universality and chaos [Asarin, 1995], necessary conditions for 
universality [Delvenne et al., 2004], the computability of entropy [Koiran, 2001], 
understanding of edge of chaos [Legenstein and Maass, 2007], and relations with 
the shadowing property [Hoyrup, 2006]. 

3.6 Embedding Turing machines in continuous time 

The embedding of Turing machines in continuous dynamical systems is often 
realized in two steps. Turing machines are first embedded into analog space 
discrete time systems, and then the obtained systems are in turn embedded into 
analog space and time systems. 

The first step can be realized with low dimensional systems with simple dy- 
namics: [Moore, 1990], [Ruohonen, 1993], [Branicky, 1995b], [Ruohonen, 1997b] 
consider general dynamical systems, [Koiran et al., 1994] piecewise affine maps, 
[Siegelmann and Sontag, 1995] sigmoidal neural nets, [Koiran and Moore, 1999] 
closed form analytic maps, which can be extended to be robust [Graca et al., 2005], 
and [Kurganskyy and Potapov, 2005] one dimensional very restricted piecewise 
defined maps. 

For the second step, the most common technique is to build a continuous time 
and space system whose discretization corresponds to the embedded analog space 
discrete time system. 

There are several classical ways to discretize a continuous time and space sys- 
tem: see Figure 4. One way is to use a virtual stroboscope: the flow x t = (f>(t, x), 
when t is restricted to integers, defines the trajectories of a discrete time dynam- 
ical system. Another possibility is through a Poincare section: the sequence x t 
of the intersections of trajectories with, for example, a hypersurface can provide 
the flow of a discrete time dynamical system. See [Hirsch et al. , 2003] . 

The opposite operation, called suspension, is usually achieved by extend- 
ing and smoothing equations, and usually requires higher dimensional systems. 
This explains why Turing machines are simulated by three-dimensional smooth 
continuous time systems in [Moore, 1990], [Moore, 1991], [Branicky, 1995b] or by 
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Fig. 4. Stroboscopic map (on left) and Poincare map (on right) of the dynamic of a 
continuous time system. 

three-dimensional piecewise constant differential equations in [Asarin et al., 1995], 
while they are known to be simulated in discrete time by only two-dimensional 
piecewise affine maps in [Koiran et al., 1994]. It is known that two-dimensional 
piecewise constant differential equations cannot 12 simulate arbitrary Turing ma- 
chines [Asarin et al., 1995], while the question whether one-dimensional piece- 
wise affine maps can simulate arbitrary Turing machines is open. Other simu- 
lations of Turing machines by continuous time dynamical systems include the ro- 
bust simulation with polynomial ODEs in [Graca et al., 2005], [Graga et al., 2007]. 
This result is an improved version of the simulation of Turing machines with real 
recursive functions in [Campagnolo et al., 2000], where it is shown that smooth 
but non-analytic classes of real recursive functions are closed under iteration. 
Notice that while the solution of a polynomial ODE is computable on its maxi- 
mal interval of existence (see Section 3.3), the simulation result shows that the 
reachability problem is undecidable for polynomial ODEs. 

In addition to Turing machines, other discrete models can be simulated 
by differential equations. Simulating two-counter machines can be achieved in 
two dimensions, or even one dimension, at the cost of a discontinuous ODE 
[Ruohoncn, 1997b]. Simulating cellular automata can be done with partial dif- 
ferential equations defined with C°° functions [Omohundro, 1984]. 

Notice that reversible computations of Turing machines (or counter machines, 
or register machines) can be simulated by ODEs with backward-unique solutions 
[Ruohonen, 1993]. 

Continuous time dynamical systems can in turn be embedded into other con- 
tinuous time systems. For example, [Maass et al., 2007] proves that a large class 
S n of systems of differential equations are universal for analog computing on 
time-varying inputs in the following sense: a system of this class can reply to 



See also already mentioned generalizations of this result in 
[Ceraens and Viksna, 1996] and [Asarin et al., 2001]. 
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some external input u(t) with the dynamics of any n" 1 order differential equation 
of the form z^ n \t) — G(z(t), z'(t), ■ ■ ■ , z^™ -1 ' (t)) + u(t) , if a suitable memoryless 
feedback and readout functions are added. As the n th order differential equa- 
tion above can simulate Turing machines, systems from S n have the power of a 
universal Turing machine. But since G is arbitrary systems from S n can actu- 
ally simulate any conceivable continuous dynamic response to an input stream. 
Moreover, this results holds for the case where inputs and outputs are required 
to be bounded. 

3.7 Discussion issues 

The key technique in embedding the time evolution of a Turing machine in a 
flow is to use "continuous clocks" as in [Branicky, 1995b]. 13 

The idea is to start from the function / : R — > R, preserving the integers, 
and build the ordinary differential equation over R 3 

y[ = c(f(r(y 2 )) - yi) 3 0(sin(27n/ 3 )) 
y' 2 = c(r(j/i) - t/ 2 ) 3 0(- sin(27n/ 3 )) 

2/3 = 1- 

Here r(x) is a rounding-like function that has value n whenever x G [n— 1/4, n + 
1/4] for some integer n, and 9{x) is for x < 0, exp(— 1/x) for x > 0, and c is 
some suitable constant. 

The variable y 3 — t is the time variable. Suppose yi(0) — 2/2(0) = x G N. For 
t € [0,1 /2], y' 2 = 0, and hence yi is kept fixed to x. Now, if /(x) = x, then 7/1 will 
be kept to x. If /(x) 7^ x, then j/i(i) will approach /(x) on this time interval, 
and from the computations in [Campagnolo, 2001], if a large enough number is 
chosen for c we can be sure that |j/i(l/2) — f(x)\ < 1/4. Consequently, we will 
have r(yi(l/2)) = f(x). Now, for t G [1/2,1], roles are inverted: y[ = 0, and 
hence y\ is kept fixed to the value /(x). On that interval, j/2 approaches /(x), 
and r (7/2(1)) = /(x). The equation has a similar behavior for all subsequent 
intervals of the form [n, n + 1 /2] and [n + 1/2, n + 1]. Hence, at all integer time 
t, /'*'(x) = r(yi(t)). 14 [Loff et al., 2007a] proposes a similar construction that 
returns / [L ' J1 (x) for all t G R. 

In other words, the construction above transforms a function over R into a 
higher dimensional ordinary differential equation that simulates its iterations. 
To do so, #(sin(27ry3)) is used as a kind of clock. Therefore, the construction is 
essentially "hybrid" since it combines smooth dynamics with non-differentiable, 
or at least non-analytic clocks to simulate the discrete dynamics of a Turing 
machine. Even if the flow is smooth (i.e. in C°°) with respect to time, the orbit 
does not admit a tangent at every point since y\ and j/2 are alternatively con- 
stant. Arguably, one can overcome this limitation by restricting Turing machine 

13 Branicky attributes the idea of a two phase computation to [Brockett, 1989] and 
[Brockett, 1991]. A similar trick is actually present in [Ruohonen, 1993]. We will 
actually not follow [Branicky, 1995b] but its presentation in [Campagnolo, 2001]. 

14 /'*' (x) denotes de ith iteration of / on x. 
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simulations to analytic flows and maps. While it was shown that analytic maps 
over unbounded domains arc able to simulate the transition function of any 
Turing machine in [Koiran and Moore, 1999], only recently it was shown that 
Turing machines can be simulated with analytic flows over unbounded domains 
in [Graga et al., 2005]. It would be desirable to extend the result to compact 
domains. However, it is conjectured in [Moore, 1998b] that this is not possible, 
i.e. that no analytic map on a compact finite-dimensional space can simulate a 
Turing machine through a reasonable input and output encoding. 

3.8 Time and space contractions 

Turing machines can be simulated by ODEs in real time: for example, in the 
constructions we described above, the state y(T) at time T e N of the solution 
of the ordinary differential equation encodes the state after T steps of the Tur- 
ing machine. However, since continuous time systems might undergo arbitrary 
space and time contractions, Turing machines, as well as accelerating Turing ma- 
chines 15 [Davies, 2001], [Copeland, 1998], [Copeland, 2002] or even oracle Turing 
machines, can actually be simulated in an arbitrary short time. 

In the paragraphs below, we will follow Ruohoncn [Ruohoncn, 1993] who 
denotes a continuous time system by the triplet (F, n, A), where F defines the 
ordinary differential equation y' = F(y) over K", with accepting set A: some 
input x is accepted iff the trajectory starting with initial condition x crosses A. 

A machine M — (F, n, A) can be accelerated: the substitution t = e u — 1 for 
instance changes M to ((G, l),n+l,4xl) where 

£ = G(g(u),u) = F{g{u))e u and g(u) = y(e u - 1), 

yielding an exponential time acceleration. Note that the derivatives of the so- 
lution with respect to the new time variable u are exponentially larger. Fur- 
thermore, the substitution t = tan(-7ru/2) gives an infinite time acceleration, 
i.e. compresses any computation, even an infinite one, into the finite interval 
< u < 1. Now, the derivatives go to infinity during the course of computation. 

Turning to space contraction, replacing the state y{t) of the machine M. = 
(F, n, A) by r(t) — y(t)e~* gives an exponentially downscaled machine ((H, 1), m+ 
1,-ffi) where 

f t =H(r(t),t) = F(r(t)e t )e- t -r(t) 

and 

H 1 = {(e"*g,t)|g € A and t > 0}. 

Obviously, this transformation reduces exponentially the distance between tra- 
jectories, which require increased precision to be distinguished. 

15 Similar possibilities of simulating accelerating Turing machines through quantum 
mechanics are discussed in [Calude and Pavlov, 2002] . 
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Hardness results in the levels of the arithmetical or analytical hierarchy for 
several decision problems about continuous time systems are derived from sim- 
ilar constructions in [Ruohoncn, 1993], [Ruohonen, 1994], [Moore, 1996], and 
[Asarin and Maler, 1998]. Completeness results, as well as exact characteriza- 
tions of the recognition power of piecewise constant derivative systems according 
to their dimensions have been obtained in [Bournez, 1999a] and [Bournez, 1999b]. 
Notice that such phenomena arc instances of the so-called Zeno's phenomena in 
hybrid systems literature: [Alur and Dill, 1990] and [Asarin and Maler, 1998]. 

It can be observed that previous constructions yield undecidability results 
only for functions over infinite or half-open intervals, since positive reals, cor- 
responding to Turing machines integer time, are mapped to intervals of the 
form [0, 1). An analytical construction is indeed possible over a finite closed do- 
main of the form [0,1], with a function G which is continuous and bounded 
on [0, 1], but non-differentiable. It follows that the event detection problem, for 
example, is undecidable even with continuous functions over compact intervals 
[Ruohonen, 1997a]. 

Undecidability is ruled out, however, if the function G is sufficiently smooth 
(say, in C 1 ), if both G and the initial value are computable, and if a sufficiently 
robust acceptance condition is considered. Indeed, problems such as the event 
detection problem then become decidable, since the system can be simulated 
effectively [Ruohonen, 1997a]. 

Instead of embedding Turing machines into continuous dynamical systems, 
it is natural to ask if there is a better way to think about computation and 
complexity for the dynamical systems that are commonly used to model the 
physical world. We address this issue in the next section. 

4 Toward a complexity theory 

Here we discuss a number of different views on the complexity of continuous 
dynamical systems. We consider general systems and question the difficulty of 
simulating continuous time systems with a digital model. We then focus on 
dissipative systems, where trajectories converge to attractors. In particular, we 
discuss the idea that the computation time should be the natural time variable 
of the ODE. Finally, we review complexity results for more general continuous 
time systems that correspond to classes of real recursive functions. 

4.1 General continuous dynamical systems 

In [Vergis et al., 1986] it was asked if analog computers can be more efficient 
than digital ones. Vergis et al. also postulated the 'Strong Church's Thesis", 
which states that the time required by a digital computer to simulate any analog 
computer is bounded by a polynomial function of the resources used by the 
analog computer. They claim that the Strong Church's Thesis is provably true 
for continuous time dynamical systems described by any Lipschitzian ODE y' = 
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The resources used by an analog computer include the time interval of opera- 
tion, say [0,T], the size of the system, which can measured by max t£ [ 0; T] ||2/(i)|| 5 
as well as the bound on the derivatives of y. For instance, mass, time of operation, 
maximum displacement, velocity, acceleration and applied force are all resources 
used by a particle described by Newtonian mechanics [Vergis et al., 1986]. 

The claim above depends on the definition of "simulation". In the article 
[Vergis et al., 1986] it is considered that the IVP y' — f(y), y(0) — x is simulated 
if, given T and some precision e, one can compute an approximation of y(T) with 
a margin of error of at most e. Using Euler's method to solve this problem, and 
considering that the round-off error is less than a, the total error bound is given 

by 



\\y(T)-y* N \\<j 



R a 
2 + h2 



(e T X-l), (5) 



where y* N is the approximation after N steps, h is the step size, A is the Lipschitz 
constant for / on [0,T], and R = max{\\y"(t)\\,t e [0,T]}. From the bound in 
(5), Vergis et al. conclude that the number N of necessary steps in Euler's 
method is polynomial in R and K They use this fact to claim that the Strong 
Church's Thesis is valid for ODEs. However, N is exponential in T, which is 
the time of operation of this analog computer. This makes the argument in 
[Vergis et al., 1986] inconclusive, as pointed out in [Orponcn, 1997]. 

More recently, Smith discusses in [Smith, 2006] if hypercomputation is possi- 
ble with respect to the n-body problem in mechanics. In particular, he shows that 
the exponential dependence in T can be eliminated. As observed in [Smith, 2006], 
all classical numerical methods of fixed degree for solving differential equations 
suffer from the same exponential dependence in T . However, by considering a 
combination of Runge-Kutta methods with degrees varying linearly with T, it is 
possible to derive a method that only requires ./V to be polynomial in T, as long 
as the absolute value of each component of /, y, and the absolute value of each 
partial derivative of / with respect to any of its arguments, having total differ- 
entiation degree k, is in (kT)°^ [Smith, 2006]. The implications of these results 
for Strong Church's Thesis are discussed in [Smith, 2006] and [Bournez, 2006]. 

The same question can be addressed in the framework of recursive analysis. 
When / : [0, 1] x [— 1, 1] — > M. is polynomial time computable and satisfies a weak 
form of the Lipschitz condition, the unique solution y on [0, 1] of IVP y' = f(t, y), 
y(0) — is always polynomial space computable [Ko, 1983]. Furthermore, solving 
in polynomial time a differential equation with this weak Lipschitz condition 
is essentially as difficult as solving a PSPACE-complete problem, since there 
exists a polynomial time computable function / as above whose solution y is not 
polynomial time computable unless P = P SPACE [Ko, 1983], [Ko, 1991]. 

Ko's results are not directly comparable to the polynomial bound shown in 
[Smith, 2006]. In recursive analysis, the input's size is the number of bits of 
precision. If the bound on the error of the approximation of y(t) is measured in 
bits, i.e., if e = 2~ d , then the required number of steps N in [Smith, 2006] is 
exponential in d. 
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If / is analytic, then the solution of y' = f(y) is also analytic. In that 
case, timestepping methods can be avoided. That is the approach followed in 
[Miiller and Moiskc, 1993], where it is proved using recursive analysis that if / 
is analytic and polynomial time computable then the solution is also polynomial 
time computable. 

In short, although Strong Church's Thesis holds for analytic ODEs, it has 
not yet been fully proved for general systems of Lipschitzian ODEs. Hence, the 
possibility of super-polynomial computation by differential equations cannot be 
ruled out, at least in principle. For informal discussions on Strong Church's 
Thesis, refer to [Aaronson, 2005] and [MacLennan, 2001]. 

Several authors have shown that certain decision or optimization problems 
(e.g. graph connectivity, linear programming) can be solved by specific con- 
tinuous dynamical systems. Examples and references can be found in the pa- 
pers [Smith, 1998] [Vergis et al., 1986], [Brockett, 1991], [Faybusovich, 1991a], 
[Helmke and Moore, 1994], [Ben-Hur ct al., 2002] 

4.2 Dissipative systems 

We now focus on dissipative systems and review two approaches. The first is 
about neural network models, such as continuous Hopficld networks, that sim- 
ulate circuits in a non-uniform manner, and leads to lower bounds on the com- 
plexity of such networks. The second deals with convergence to attractors, and 
considers suitable energy functions as ways to measure the complexity of con- 
tinuous time processes. 

When considering dissipative systems, such as Hopfield neural networks, the 
following approach to a complexity theory is natural. Consider families (C' n ) ne pf 
of continuous time systems, for each input length n > 0. Given some digital 
input w £ {0, 1}*, the system C n evolves on input w (or some encoding of w), 
where n is the length of w. It will eventually reach some stable state, which is 
considered the result of the computation. 

This circuit inspired notion of computability is the most common in the 
literature about the computational complexity of neural networks models: see 
survey [Sima and Orponen, 2003c]. With respect to this approach, continuous 
time symmetric Hopfield networks with a saturated linear activation function 
have been proved to simulate arbitrary discrete-time binary recurrent neural 
networks, at the cost of only a linear size overhead [Orponen and Si'ma, 2000], 
[Sima and Orponen, 2003b]. This might be thought counterintuitive, since such 
symmetric networks, which are constrained by some Liapunov energy function, 
can only exhibit convergence phenomena, and hence cannot even realize a simple 
alternating bit. However, the convergence of dissipative systems can be exponen- 
tially long in the size of the system [Sima and Orponen, 2003a], and hence the 
simulation can be accomplished using a subnetwork that provides 2" clock pulses 
before converging. 

The languages recognized by polynomial size families of discrete-time Hop- 
ficld networks have been proved in [Orponen, 1996] to correspond to non-uniform 
complexity class PSPACE/poly for arbitrary interconnection weights, and to 
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P/poly for polynomially bounded weights. Therefore, families of continuous time 
symmetric Hopfield networks have the same lower bounds. However, these lower 
bounds may be not tight, since upper bounds for continuous time dynamics are 
not known [Si'ma and Orponcn, 2003b], [Si'ma and Orponen, 2003c]. 

Let us now turn our attention to dissipative systems with a Lyapunov func- 
tion E. 

Gori and Meer [Gori and Meer, 2002] consider a computational model which 
has the capability of finding the minimizers (i.e. the points of local or global 
minimum) of E. To prevent the complexity of a problem from being hidden in 
the description of E, this function must be easy to compute. In that setting, 
a problem 77 is considered easy if there exists el uniiiioclc d function E (i.e. all 
local minimizers of E are global minimizers) such that the solution of 77 can be 
obtained from the global minimum of E. 

More precisely, Gori and Meer investigate in [Gori and Meer, 2002] a model 
where a problem 77 over the reals is considered to be solved if there exist a 
family (E n ) n : R n x Ri(n) — > M of energy functions, given by a uniform family of 
straight line programs (q is some fixed polynomial), and another family (N n ) n of 
straight line programs, such that for all input d, a solution 77(d) of the problem 
can be computed using N q ^(w*), from a global minimizer w* of w — > E n (d, w). 

Gori and Meer define classes U and NU in analogy with P and NP in 
classical complexity. U corresponds to the above mentioned case where for all d, 
w — > E n (d, w) is unimodal, in opposition to NU where it needs not be unimodal. 
Notions of reductions are introduced, and it is proved that the natural optimiza- 
tion problem "find the minimum of some linear objective function over a set de- 
fined by quadratic multivariate polynomial constraints" is NU-h&rd. They show 
that there exist (artificial) NU complete problems. These ideas are generalized to 
obtain a polynomial hierarchy, with complete problems [Gori and Meer, 2002]. 

Actually, Gori and Meer's proposed framework is rather abstract, avoiding 
several problems connected to what one might expect of a true complexity theory 
for continuous time computations. Nonetheless, it has the great advantage of not 
relying on any particular complexity measure for the computation of trajectories. 
See the interesting discussion in [Gori and Meer, 2002]. 

However, one would like to understand the complexity of approaching the 
minima of energy functions, which correspond to the equilibria of dynamical sys- 
tems. First steps toward this end have been investigated in [Bcn-Hur ct al., 2002], 
where dissipative systems with exponential convergence are explored. Recall that 
if x* is a sink, then the rate of convergence toward x* satisfies 

\x(t)-x*\ = e~ xt 

where —A is the largest real part of an eigenvalue of Df(x*). This means that 
r = 1 /A is a natural characteristic time of the attractor: every r log 2 time units, 
a new bit of the attractor is computed. 

For the systems considered in [Ben-Hur et al., 2002], each sink has an at- 
tracting region, where the trajectories are trapped. One can define the com- 
putation time t c of a dissipative continuous time dynamical system as t c = 
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max(t c (e),t c (U)), where t c (e) is the time required to reach some e vicinity of 
some attractor, and t c (U) is the time required to reach its attracting region. 
Then, T = ^ is a dimensionless complexity measure, invariant under any linear 
time contraction. 

Two continuous time algorithms, MAX to compute the maximum of n num- 
bers, and FLOW to compute the maximum flow problem have been studied 
in this framework in [Bcn-Hur ct al., 2002]. MAX has been shown to belong 
to proposed complexity class CLOG (continuous log time), and FLOW to CP 
(continuous polynomial time). The authors conjecture that CP corresponds to 
classical polynomial time [Ben-Hur et al., 2002]. Both MAX and FLOW al- 
gorithms are special cases of a flow proposed in [Faybusovich, 1991b] to solve 
linear programming problems, further investigated in [Bcn-Hur et al., 2003] and 
[Bcn-Hur et al., 2004a]. Variations on definitions of complexity classes, as well 
as ways to introduce non-deterministic classes in relation to computations by 
chaotic attractors have also been discussed in [Siegelmann and Fishman, 1998]. 



4.3 Complexity and real recursive functions 



Real recursive functions are a convenient way to analyse the computational power 
of certain operations over real functions. Additionally, given a continous time 
model, if its equivalence with a function algebra of real recursive functions can 
be established, then some properties of the model can be proved inductively over 
the function algebra. 

Since many time and space complexity classes have recursive characteriza- 
tions over N [Clote, 1998], structural complexity results about discrete operations 
may imply lower and upper bounds on the computational complexity of real re- 
cursive functions. This approach was followed in [Campagnolo ct al., 2002] to 
show that C contains extensions of the elementary functions, and further devel- 
oped in [Campagnolo, 2004] to obtain weaker classes that relate to the exponen- 
tial space hierarchy. This tells us something about the computational complexity 
of certain dynamical systems. For instance, C corresponds to cascades of finite 
depth, each level of which depends linearly on its own variables and the output 
of the level before it. 

Results about the idea of lifting computability questions over N to 1 have 
been discussed before. Concerning complexity, the question P = NP in clas- 
sical complexity has been investigated using real recursive functions by Costa 
and Mycka. In particular, they propose two classes of real recursive functions 
such that their inequality would imply P ^ NP in [Costa and Mycka, 2006], 
[Mycka and Costa, 2006]. More generally a part of Costa and Mycka's program, 
explicitly stated in [Mycka and Costa, 2005] and [Mycka and Costa, 2007], uses 
recursion theory on the reals to establish a bridge between computability and 
complexity theory and mathematical analysis. 



24 



5 Noise and Robustness 

Up to this point we have considered continuous time computations in idealized, 
noise- free spaces. As was also the case in the survey by Orponen [Orponen, 1997], 
most of the results we discussed disregard the impact of noise and imprecision in 
continuous time systems. This is a recurrently criticized weakness of the research 
in the field. While there have not been major breakthroughs with regards to 
these problems as they relate specifically to continuous time computations, some 
interesting developments concerning noise and imprecision have come about in 
discrete time analog computation studies. In this section we will broaden our 
scope to discuss a number of discrete time results. We believe that some of these 
studies and results might be generalized to, or at least provide some insight into, 
the effects of noise and imprecision on continuous time systems, although this 
work has yet to be done. 

We first focus on systems with a bounded state space about which a folklore 
conjecture claims that robustness implies decidability. We review some results 
that support this conjecture as well as others that challenge it. At the end of 
this section we discuss continuous time systems with unbounded state spaces. 

Common techniques to simulate Turing machines by dynamical systems in 
bounded state spaces require the encoding of the configuration of the Turing ma- 
chine into real numbers. Since Turing machines have unbounded tapes (otherwise 
they would degenerate into finite automata), these simulations are destroyed if 
the real numbers or the functions involved are not represented with infinite 
precision. This leads to the folklore conjecture, popular in particular in the ver- 
ification community, which states that undecidability do not hold for "realistic", 
"unprecise", "noisy", "fuzzy", or "robust" systems. See for example [Franzle, 1999], 
[Foy, 2004] for various statements of this conjecture, and [Asarin, 2006] for dis- 
cussions on other arguments that lead to this conjecture. 

There is no consensus on what a realistic noise model is. A discussion of this 
subject would require to question what are good models of the physical world. 
In the absence of a generally accepted noise model, one can however consider 
various models for noise, imprecision or smoothness conditions, and investigate 
the properties of the resulting systems. 

In particular, there have been several attempts to show that noisy ana- 
log systems are at best equivalent to finite automata. Brockett proved that 
continuous time dynamical systems can simulate arbitrary finite automata in 
[Brockett, 1989]. Using topological arguments based on homotopy equivalence 
relations and associated Deck transformations, he showed in [Brockett, 1994] 
that some automata can be associated to dissipative continuous time systems. 

Maass and Orponen proved that the presence of bounded noise reduces the 
power of a large set of discrete time analog models to that of finite automata 
in [Maass and Orponen, 1998]. This extends a previous result established in 
[Casey, 1996], [Casey, 1998] for the case where the output is assumed to be per- 
fectly reliable (i.e. p— 1/2 in what follows). 

Maass and Orponen's idea is to replace a perfect discrete time dynamic of 
type x i+ i = f(xi,di), where Oj is the symbol input at time i, over a compact 
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domain, by a probabilistic dynamic 



Probability(x i+1 £ B) = / z(f(x l ,a i ),q)dp, (6) 

JqeB 

where B is any Borel set. Here, z is a density kernel reflecting arbitrary noise, 
which is assumed to be piecewise equicontinuous. This means that, for all e, 
there exists 5 such that for all r,p,q, \\p — q\\ < 5 implies \z{r,p) — z(r,q)\ < e. 
They denote by n x (q) the distribution of states after string x is processed from 
some fixed initial state q, and they consider the following robust acceptance 
condition: a language L is recognized, if there exists p > such that x £ L iff 
J F T xu (q)dp > 1/2 + p for some u £ {U}* , and x £ L iff J p TT xu (q)djjL < 1/2 — p 
for all u e {U}* , where [/ is the blank symbol, and i 7, the set of accepting states. 
Then, they show that the space of functions tt x (.) can be partitioned into finitely 
many classes C such that two functions n x (.) and n y (.) in the same class satisfy 
f r \ n x( r ) — Tt y (r)\d(i < P- Therefore, two words x,y in the same class satisfy 
xw £ L iff yw £ L for all words w. 

In fact, for any common noise, such as Gaussian noise, which is nonzero on a 
sufficiently large part of the state space, systems described by (6) are unable to 
recognize even arbitrary regular languages [Maass and Sontag, 1999]. They rec- 
ognize precisely the definite languages introduced by [Rabin, 1963], as shown 
in [Maass and Sontag, 1999] and [Ben-Hur et al., 2004b]. If the noise level is 
bounded, then all regular languages can be recognized [Maass and Orponcn, 1998] 
Feedback continuous time circuits in [Maass et al., 2007] have the same compu- 
tational power when subject to bounded noise. 

As an alternative to the probabilistic approach of Maass and Orponen, noise 
can be modeled through non-determinism. One can associate to deterministic 
noise-free discrete time dynamical system S defined by x i+x = f(x.i), the non- 
deterministic e-perturbated system S e whose trajectories are sequences (x n ) n 
with \\x i+ i — f(xi) || < e. For a dynamical system S, it is natural to consider the 
predicate Reach[S](x,y) (respectively Reach n [S\(x,y)), which is true if there 
exists a trajectory of S from x to y (resp. in i < n steps). Then, algorith- 
mic verification of safety of state properties is closely related to the problem 
of computing reachable states. Given S, and a subset of initial states So, let 
Reach[S] denote the set of y's such that Reach[S](x, y) for some x £ So- Given 
a state property p (i.e. a property which is either true, or false in a state s), 
let [[-<p]\ denote the subset of states s where p is false. Then S is safe (p is 
an invariant) iff Reach[S] n [[->p]] — (see, for example, [Alur et al., 1995] and 
[Nicollin et al., 1993]). 

If the class of systems under consideration is such that relation Reach n [S](x, y) 
is recursive 16 (assuming that So recursively enumerable), then Reach[S] is re- 
cursively enumerable because Reach[S] — (J n Reach n [S] . Several papers have 
been devoted to prove that Reach[S] is indeed recursive for classes of dynamical 
systems under different notions of robustness. We now review several of them. 



Recursive in x, y and n. 
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Franzle observes in [Franzle, 1999] that the computation of Reach[S e ] by 
Reach[S e ] = [J Reach n [S e ] must always terminate if Reach[S e ] has a strongly 
finite diameter. This means that there exists an infinite number of points in 
Reach[S e ] at mutual distance of at least e, which is not possible over a bounded 
domain. It follows that if we call robust a system which is either unsafe, or 
whose e-perturbated system is safe for some e, then safety is decidable for robust 
systems over compact domains [Franzle, 1999]. 

Consider as in [Puri, 1998] the relation Reach u [S] — f} e>0 Reach[S e ], corre- 
sponding to states that stay reachable when noise converges to 0. Asarin and 
Bouajjani prove in [Asarin and Bouajjani, 2001] that for large classes of discrete 
and continuous time dynamical systems (Turing machines, piecewise affine maps, 
piecewise constant differential equations), Reaches] is co-recursively enumer- 
able. Furthermore, any co-recursively enumerable relation is of form Reaches] 
for some S for the classes that Asarin and Bouajjani consider. Therefore, if we 
call robust a system such that Reach[S] — Reaches], then computing Reach[S\ 
is decidable for robust systems. 

Asarin and Collins considered in [Asarin and Collins, 2005] a model of Turing 
machines exposed to a small stochastic noise, whose computational power have 
been characterized to correspond to 77° . It is interesting to compare this result 
with previous results where a small non-deterministic noise lead to 77° (co- 
recursively enumerable sets) computational power only. 

We now turn our attention to results that challenge the conjecture that ro- 
bustness implies decidability. A first example is that the safety of a system is 
still undccidable if the transition relation of the system is open, as proved in 
[Hcnzingcr and Raskin, 2000], and [Asarin, 2006]. However, the question for the 
restriction to a uniform non-deterministic noise bounded from below is open 
[Asarin, 2006]. 

Noise can also be modeled by perturbating trajectories. Gupta, Hcnzingcr 
and Jagadeesan consider in [Gupta et al., 1997] a metric over trajectories of 
timed automata, and assume that if a system accepts a trajectory, then it must 
accept neighboring trajectories also. They prove that this notion of robustness is 
not sufficient to avoid undecidability of complementation for Timed automata. 
Hcnzingcr and Raskin prove in [Henzinger and Raskin, 2000] that major unde- 
cidability results about verification of hybrid systems are still undecidable for 
robust systems in that sense. 

Finally, we review a recent robustness result for continuous time dynamical 
systems with unbounded state space. Graga, Campagnolo and Buescu prove in 
[Graca et al., 2007] that polynomial differential equations can simulate robustly 
Turing machines in real time. More precisely, let us consider that 9 : N 3 — > N 3 is 
the transition function some Turing machine M whose configuration is encoded 
on N 3 . Then, there is a e > 0, a solution / of a polynomial ODE, and an initial 
condition /(0) such that the solution of y' = f(t, y) encodes the state of M after 
t steps with error at most e. Moreover, this holds for a neighborhood of any 
integer t even if / and the initial condition /(0) are perturbed. Obviously, this 
kind of simulation requires the system to have an unbounded state space. 
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6 Conclusion 



Having surveyed the field of continuous time computation theory, we see that 
it provides insights into many diverse areas such as verification, control the- 
ory, VLSI design, neural networks, analog machines, recursion theory, theory of 
differential equations and computational complexity 

We have attempted to give a systematic overview of the most relevant models 
and results on continuous time computations. In the last decade many new results 
have been obtained, indicating that this is an active field of research. We reviewed 
recent developments of the theory of continuous time computation with respect 
to comput ability, complexity and robustness to noise, and we identified several 
open problems. To conclude, we will discuss some directions for future research 
related to these areas. 



Computability. It is not clear if a unifying concept similar to the Church- 
Turing thesis exists for continuous time computation. Although it has been 
shown that some continuous time models exhibit super Turing power, these 
results rely on the use of an infinite amount of resources such as time, space, 
precision, or energy. In general, it is believed that "reasonable" continuous time 
models cannot compute beyond Turing machines. This raises the question if 
physically realistic continuous time computation can be as powerful as digital 
computation. We saw that if we restrict continuous time systems to evolve in a 
bounded state space and to be subjected to noise, then they become compara- 
ble to finite automata. However, with a bounded state space, Turing machines 
also degenerate into finite automata. Since analytic and robust continuous time 
systems can simulate Turing machines in an unbounded state space, we believe 
that digital computation and analog continuous time computation are equally 
powerful from the computability point of view. Moreover, as we saw, several 
recent results establish the equivalence between functions computable by poly- 
nomial ODEs, GPAC-computable functions and real computable functions in 
the framework of recursive analysis. These kind of results reinforce the idea that 
there could be an unified framework for continuous time computations, analo- 
gous to what occurs in classical computation theory. 

We feel that a general paradigm of realistic continuous time computations 
ideally should only involve analytic functions, since these are often considered 
as the most acceptable from a physical point of view. Continuous dynamical 
systems are a natural form of representing continuous time processes. Classical 
systems like the van der Pol equation, the Lotka-Volterra system or the Lorenz 
equations are described with differential equations with an analytic, even poly- 
nomial, right-hand side. These physics-related arguments combined with the 
computability properties of systems of polynomial differential equations lead us 
to suggest that this continuous time model is a possible candidate for a general 
paradigm of continuous time computation. We believe that this idea deserves 
further investigation. 
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Complexity. We saw that a complexity theory for continuous time compu- 
tation is still under way and that there has not been an agreement between 
authors on basic definitions such as computation time or input size. The results 
described in Section 4 are either derived from concepts which are intrinsic to the 
continuous time systems under study or related to classical complexity theory. 
As computable analysis is a well established and understood framework for the 
study of computational complexity of continuous time systems, we believe that 
understanding relations between different approaches and computable analysis 
from a complexity point of view is of first importance. There are still many open 
questions about upper bounds for continuous time models. For example, upper 
bounds are not known for Hopfield networks and general systems of Lipschi- 
tizian ODEs, which compromises the validity of the Strong Turing thesis. We 
saw that this thesis might hold for systems of analytic ODEs. This leads us to 
ask if a continuous time computation theory based on polynomial ODEs could 
be naturally extended to a complexity theory. 

Computable analysis also permits to study the complexity of real recursive 
functions. One of the most intriguing area of research in continuous time com- 
putation tries to explore the link between real recursive functions and com- 
putational complexity to establish a translation of open problems of classical 
complexity into Analysis. 

Robustness. We saw that very little research has been done with respect to the 
robustness and tolerance to noise of continuous time systems. One might ask how 
the power of analog computations increases with their precision. This question 
was raised and formalized for discrete time analog systems, in particular for dy- 
namical recognizers, in [Moore, 1998b] but most of the research in that direction 
has yet to be done. Many interesting open questions arise if one asks if undecid- 
ability results for continuous time systems still hold for robust systems. This is 
of first importance for example for the verification of hybrid systems, since this 
question is closely related to the question of termination of automatic verification 
procedures. A better understanding of the hypotheses under which noise yield 
decidability or undecidability is required. For example, non-deterministic noise 
on open systems does not rule out undecidability, but the question is unanswered 
for a uniform noise bounded from below [Asarin, 2006]. 
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